1. Field Of The Invention
The present invention concerns a digital video editor which converts frames of analog video information into corresponding frames of digital video information in real time and which detects and fills in lost frames of video information by re-reading the original analog information in additional passes or by reconstructing them from adjacent frames.
2. Description Of The Related Art
With recent advances in speed and storage capacity of personal computing systems, it has become possible to convert analog video information in real time into corresponding digital video information, to edit the information while in digital form, and to convert the edited digital information back into an analog form.
Such an arrangement is desirable for several reasons. First, it eliminates the need for customized analog video editing machinery and instead allows a general purpose programmable computer to perform editing. The editing program on such a computer may be changed at will thereby permitting flexibility and facilitating change when new advances in editing technology are made.
Second, digital editing offers a simpler interaction interface and permits relatively unskilled workers to produce sophisticated results. As skill grows, a more complete set of editing effects may be generated through use of computer graphics, digital image processing and text editing techniques.
Finally, as the cost of personal computing machinery decreases, it becomes possible for more and more people to use video editing capabilities. Those capabilities would be foreclosed to an ordinary individual who could not afford high-end and single use items such as analog editing machinery.
However, acquiring digital video information that is full frame size, full color, and full motion places unprecedented demands on the bandwidth of digital computing machinery, and these demands can cause entire frames of video information to be lost. For example, corresponding to the NTSC standard, a full frame is 640.times.480 pixels, and full motion is 30 frames per second. With full color of 24 bits per pixel, the bandwidth requirements to handle such a digital video signal in real time is 640.times.480.times.30.times.24=221,184,000 bits per second which far exceeds the capabilities of currently available personal computing machinery. Future standards such as HDTV (high definition television) are even more demanding. Compression techniques such as JPEG ("joint photographic expert group"), MPEG ("motion picture expert group") and CCITT H.261 ("International Telegraph and Telephone Consultative Committee") have been proposed to reduce the bandwidth requirements to acceptable levels, but other factors can still cause video frames to be lost.
For example, known compression techniques are not "loss-less". That is, known compression techniques introduce errors when the compressed data are uncompressed. If the final video is to be of the same quality as the original video, compression cannot be so great as to introduce errors that are perceived as visual degradation in image quality. This limits the amount of bandwidth reduction obtainable from compression, and a fine balance must be struck between bandwidth reduction from compression and the bandwidth capabilities of the digital computing machinery. Today's high-end PCs and workstations have barely the bandwidth capabilities required to handle NTSC-type data, and HDTV processing put those capabilities into jeopardy.
Another factor contributing to the loss of video frames is multitasking. Current operating systems on workstations as well as next generation operating systems on personal computing equipment have multitasking capabilities which permit several tasks to run concurrently. Those several tasks compete in real-time for shared resources of the computing equipment, such as an internal computing bus, the CPU and the hard disk. Multitasking, however, places further bandwidth demands on the digital equipment, as well as raises the possibility that the CPU is busy servicing one task when a video frame is ready to be stored. If the CPU is not able to store the video frame before the next video frame becomes available, that video frame will be ignored and lost.
Lost frames of digital video information are not acceptable for many reasons. Lost frames can make it impossible to obtain high quality cuts, merges, compositions, special effects, etc. For example, when creating a slow motion special effect, missing frames cause large jumps and unevenness that are visually unacceptable. Further, errors caused by missing frames accumulate over longer clips. Even if only three frames are lost per minute, an entire second of material is lost in a ten minute sequence. Because sound is generally handled on a separate channel from video, such losses will quickly de-synchronize sound and video.
Further still, some compression techniques such as MPEG do not fully encode each video frame independently, but rather fully encode only some reference frames. Frames between the reference frames are encoded as the difference or change from the reference frames. If a reference frame is lost, it is impossible to re-create the original image for frames which depend on the lost reference frame.
To handle lost frames of digital video information in a real time digital video environment, a two-pass procedure has been proposed. In the first pass, the analog video equipment plays out the analog video signal, and the digital video equipment digitizes and stores as many frames of video information as possible. While the analog video equipment is playing out the video signal, it generates frame number information which is transmitted to the digital equipment over a frame number interface that is distinct from the video interface. The digital video equipment inspects the frame number information to determine if any frames have been lost and to determine the identity of those frames. If any frames have been lost, then a second pass is performed in which the analog video equipment is repositioned using the frame number information from the frame number interface and the missing frames are played out. If frames are still missing after the second pass, then additional passes are performed until all frames have been acquired in digital form.
The foregoing procedure is not entirely satisfactory. First, because the frame number information is not part of the video signal, and is supplied on a frame number interface that is separate from the video interface, it is possible that the video information on the video interface is not synchronized with the frame number information on the frame number interface. This means that the frame on the video interface is different from the frame indicated by the frame number information. Accordingly, it is not always possible to ensure that missing frames are properly detected, or to ensure that properly detected missing frames are supplied with the needed video data.
Second, there is no guarantee that progress will be made in finding missing frames. More specifically, even in the case where it is known with certainty that a frame is missing, there is no guarantee that the particular frame in question can be captured and added to the collection of already-digitized video frames. In fact, the same problems that caused the frame to be missed in the first pass, such as multitasking demands on shared resources, may also exist on subsequent passes, making it difficult ever to capture the missing frame.